Representing Narrative in Multimedia Systems
نویسنده
چکیده
This report investigates the hypothesis: patterns in multimedia discourses can be used to specify and extract machine-executable representations of media content that capture important elements of stories. Multimedia Information Systems are amongst the more pervasive of information systems. The physical and conceptual task of synthesising two or more modalities of communication-visual and linguistic, to produce information, to convey 'emotions' and to convey instruction is a complex one. The converse task of extracting information, identifying emotional states and locating instruction is perhaps even more complex. This report considers the structure of multimedia data: we consider multimedia data as having an underlying story or narrative. We are synthesising notions from narrative theory, corpus linguistics and Language for Special Purpose (LSP) and case studies are considered for these notions. Firstly, a graphical display of narrative theory and narrative structural phenomena is illustrated as a case study for modelling narrative, and a notation for dealing with 'simple' narrative elements (events, states and existents) is developed. Then, random film collateral texts – Audio Description and Screenplay film scripts-are analysed in terms of mental states or emotions to see whether these states are indicatory of narrative elements and whether they can be extracted automatically. We then try to extract repeating lexical patterns from films to examine whether we can develop a local grammar for film using two film script corpora as a case study. A method is discussed for extracting qualitative information from linguistic symbols. Informed by ideas of special language, where trained experts communicate their needs in a specialised vocabulary, film collateral texts (film scripts) are analysed in terms of established corpus linguistic analysis methods to elicit information about frequent words and their collocates. From this analysis evidence of local grammars emerge as well as repeating surface patterns indicating what is believed to be important narrative elements. We believe that further study of the corpora will reveal a local grammar for the collateral text and cues for important natural elements that can be used to populate a machine-executable representation of multimedia data.
منابع مشابه
Linking Video and Text via Representations of Narrative
The ongoing TIWO project is investigating the synthesis of language technologies, like information extraction and corpus-based text analysis, video data modeling and knowledge representation. The aim is to develop a computational account of how video and text can be integrated by representations of narrative in multimedia systems. The multimedia domain is that of film and audio description – an...
متن کاملNarrative-driven Multimedia Tagging and Retrieval: Investigating Design and Practice for Speech-based Mobile Applications
This paper presents a design concept for speech-based mobile applications that is based on the use of a narrative storyline. Its main contribution is to introduce the idea of conceptualizing speech-based mobile multimedia tagging and retrieval applications as a story that develops via interaction of the user with characters representing elements of the system. The aim of this paper is to encour...
متن کاملSupporting Oral Narrative Development of Kindergarten English Language Learners Using Multimedia Stories
Narrative ability comes before literacy for bilingual students and helps narrow down the gap in text-level literacy between English language learners (ELLs) and native English speakers. Kindergarten ELLs are the best age group to receive intervention to improve their oral narrative skills. Multimedia stories have potential to assist kindergarten ELLs in developing oral narrative skills, but in ...
متن کاملAn Investigation into Automatically Captured Autobiographical Metadata, and the Support for Autobiographical Narrative Generation
FACULTY OF ENGINEERING, SCIENCE AND MATHEMATICS SCHOOL OF ELECTRONICS AND COMPUTER SCIENCE INTELLIGENCE, AGENTS, MULTIMEDIA GROUP A mini-thesis submitted for transfer from MPhil to PhD by Mischa M Tuffield Personal information and the act of publishing multimedia artifacts to the World Wide Web is becoming more and more observable. This report presents an infrastructure for the capturing and ex...
متن کاملTransmedia Storytelling: Implicit Consumers, Narrative Worlds, and Branding in Contemporary Media Production
Many concepts have been developed to describe the convergence of media, languages, and formats in contemporary media systems. This article is a theoretical reflection on “transmedia storytelling” from a perspective that integrates semiotics and narratology in the context of media studies. After dealing with the conceptual chaos around transmedia storytelling, the article analyzes how these new ...
متن کامل